The success of digital imaging and video has lead to a wide use of this technology in many fields of everyday life. Technology to edit, alter or modify digital images or video sequences is commercially available and allows modifications of the contents of said images or videos without leaving traces. For a variety of applications, such as evidential imaging in law enforcement, medical documentation, damage assessment for insurance purposes, etc., it is necessary to ensure that an image or video has not been modified and is congruent with the image or video originally taken. This led to the development of image or video authentication systems for which an example is shown in FIG. 1, wherein a signature or a watermark is created at 1.20 for a digital signal, i.e. an image or video, which is acquired in 1.10. The signature is embedded at 1.30 in the digital image or video. Thereafter the image or video is processed or tampered in 1.40, played, recorded or extracted in 1.50 and finally verified in 1.60 in order to either ensure that the authenticity of the digital image or video is proven or that modifications of the digital image or video are revealed.
In certain situations some changes to images are desired and allowable and should not be classified as malicious tampering when validating the authenticity of the images/video. Such changes occur e.g. when applying lossy compression to the digital image in order to reduce storage capacity or increase transmission rate. Lossy compression causes image modifications, but not to an extent that degrades the intended use of the images. An example for such a compression technique is the JPEG image file format, which reduces the size of a digital image considerably, i.e. the bit and byte sequence of the image is modified, while the perceptual information of the image is maintained.
Therefore a need exists for image authentication which distinguishes between allowable image modifications, such as lossy compression, and malicious tampering, such as the replacement of an image area with new content or with content copied from an earlier or later point in time of the same scenery.
One approach to authenticate images is to use classical cryptography, whereby a digital image is converted to a hash using a cryptographic key. The generated hash is taken as a “fingerprint” of the digital image. A digital image which authenticity is to be validated is converted to a hash using the same cryptographic key. If the new hash is exactly the same as the originally generated hash, the authenticity of the image is validated. By its nature classical cryptography is bit sensitive and a change of one bit in the original digital signal results in a completely different hash. Thus, when one bit of the image to be validated is changed during e.g. transmission or storage by e.g. compression, the image to be validated is classified as being tampered. Thus classical cryptography is not suited for authentication of a digital image having the above requirements concerning allowable modifications of the image.
An alternative is the embedding of semi-fragile watermarks or the creation of robust digital signatures. Both concepts maintain the perceptual information of the image and are based on generating additional information from the digital image and hiding the information in the image itself or its framework, or by transmitting or storing the additional information separately as “meta-data” with the image.
Semi-fragile embedded watermarks for authentication purposes provide tolerance against allowable operations such as compression at modest compression rates. However, when the digital signal has been tampered, watermark detection fails in areas of the original signal which have been tampered. The embedding of semi-fragile watermarks typically fails to provide the ability to distinguish between innocuous and malicious signal modifications. Furthermore it is fragile because the watermarks typically cannot survive high compression ratios. Also, in certain cases such as flat regions, a watermark cannot be embedded. Finally, it is not possible to identify tampering of the digital signal when flat contents is inserted in the image during tampering.
Robust signatures are a set of bits which summarizes the content of the image and which is relatively unchanged by compression or other allowable operations, but altered considerably by tampering. Many image properties can be used for computing a signature, e.g. edges, moments, DC-values, histograms, compression invariants, and projections onto smoothed noise patterns. All methods of generating signatures have in common that the size of the signature increases rapidly with the level of protection, i.e. the ability to accurately localize tampering. This poses a problem due to storing and transmitting requirements because when embedding the signature into the digital image, the size of a signature is critical, especially when embedding the signature as a robust watermark. A robust watermark is defined as a watermark which allows correct extraction of the payload bits even after operations that significantly degrade or damage the image, such as heavy compression or tampering by e.g. replacement of some pixels. A robust watermark allows in contrast to semi-fragile watermarks to extract the payload correctly, even after tampering. However, the payload of robust watermark schemes is very limited, typically just tens of bits. Thus, the problem to be solved by the invention is defined as how to provide a robust tamper detection for an audio-visual signal such as a digital image or a video, allowing localisation of tampering in the signal, but adding little payload to the signal.